[r3.6] execution, db: bind StateCache fills to transaction views and reject stale fills - #23031

…stale fills (#22444)

Closes #22356 — the forward stale-fill direction: a fill from a read
view older than the latest apply is rejected. The reverse direction
around unwind is tracked in #22463 and stays open (details in the scope
section).

`StateCache` is one process-global in-memory cache of the **latest
committed values** of accounts, storage and code. It exists to skip the
file-accessor/MDBX stack on repeated `GetLatest` reads. It is not a
snapshot: per key it holds one value — the newest committed value known
to the process, whether applied by execution or filled from a read. It
is on by default; `USE_STATE_CACHE=false` constructs no cache at all
(readers go straight to the backing tx), and `STATE_CACHE_FILLS=false`
disables every reader write (apply-only mode) — the kill switch and the
A/B lever for measuring what fills contribute.

| Actor | Reads | Writes | When |
|---|---|---|---|
| Canonical execution (`SharedDomains.Commit`) | via `GetLatest` |
**apply**: put committed updates, physically delete deletions | after
the RwTx commit succeeds — `sd.Commit` walks the pending updates into
the cache only once `tx.Commit` returns |
| Embedded RPC (`execmodule.CacheView` → `SharedDomains.AsGetter`) | via
`GetLatest` | **fill**: on a miss, offer the value read from its own tx
| during any request at `latest` |
| Read-ahead warmup (`blocks_read_ahead`) | — | **fill**: same, to
pre-warm keys for exec | while exec runs |

A **fill** is a cache write performed on behalf of a *database* reader
(fill-on-miss); an **apply** is the authoritative post-commit write from
`SharedDomains.Commit`. A **read view** is what a read-only temporal tx
sees (its MDBX read tx + pinned files view); its **frontier** is the
exclusive txNum end of what it can see — a view with frontier N sees
txNums < N. This sharing — including the RPC fills — is `main`'s code
today (wiring: `node/eth/backend.go` hands `execmodule.Cache` to the
embedded rpcdaemon as its kvcache; `CacheView.Get` →
`SharedDomains.AsGetter` → `getLatestMetered`, which consults and fills
the shared cache). The PR does not add the sharing; it adds the missing
ordering.

1. **Reader** begins a read-only tx; its frontier is N.
2. **Exec** at txNum M >= N deletes key K and commits; the post-commit
apply empties K's slot.
3. **Reader** reads K: cache miss → reads its own older tx → gets the
pre-delete value → its fill re-inserts it (the slot is empty, so
`PutIfAbsent` lands).
4. Anyone's next `GetLatest(K)` serves the deleted value from the cache.

Each step is consistent inside its own tx. The bug is the unordered
hand-off between two txs through the shared cache — no per-tx
consistency property can prevent it.

```mermaid
sequenceDiagram
    participant Reader as Reader with frontier N
    participant Cache as shared StateCache
    participant Exec as Exec commit
    Exec->>Cache: apply at txNum M >= N. delete K, appliedEnd = M+1
    Reader->>Cache: GetLatest(K). miss, the slot is empty
    Reader->>Reader: falls back to its own older tx. pre-delete value
    Reader->>Cache: fill(K, pre-delete value)
    Note over Cache: main. PutIfAbsent lands, K is resurrected
    Note over Cache: this PR. rejected, frontier N < appliedEnd
```

`StateCache` has no data methods (like `db`). Access goes through two
handles (like `tx`):

- **`ReadView`** — reads plus admission-gated fills, bound to one tx's
read view; it must not outlive the tx. `SharedDomains.AsGetter*(tx)`
builds one per getter; the plain `GetLatest` wrappers use a
frontier-less view and bind a frontier only on the miss path. The view
resolves its frontier internally, so a fill can no longer be paired with
another tx's frontier.
- **`Applier`** — `Apply` / `Unwind` / `Clear`. Used by the
authoritative `SharedDomains` commit/unwind paths (every SD holds a
handle; the operations serialize under the admission lock).

The cache tracks, per domain, the exclusive end of what has been applied
(`appliedEnd`), guarded by one `RWMutex`:

- `Applier.Apply` (write lock) advances `appliedEnd` and mutates the
cache in the same critical section. Applying txNum X records `appliedEnd
= X+1`.
- `ReadView.Fill` (read lock) rechecks its view's frontier and inserts
before releasing: admitted iff `frontier >= appliedEnd`. Fills are
put-if-absent, so they never replace a live authoritative entry.

Ordering proof, two cases: if the fill takes the lock first, the later
apply overwrites or deletes it; if the apply takes the lock first, the
stale fill is rejected. Equality is safe because a view whose frontier
equals the applied end has seen the apply. Checking freshness only at
view creation would not be enough — an apply can land between the check
and the fill — which is why admission happens at fill time, under the
same lock applies take.

`DomainVisibleEnd` reports the exact exclusive frontier of a tx's domain
read view: a file covering `[0,N)` reports N; a hot-DB view containing
txNum N reports N+1. Views without an exact frontier answer `ok=false`
and never fill — remote and history-disabled backends, and
dependency-clamped values views (their reads mix ages, and the
clamped-away state can appear later without any apply); such reads still
work, they just skip cache population. Two memos, one per path:
read-only temporal txs memoize the frontier per domain in a tx-local
cache, reset only by `ForceReopenUnderlyingFilesTx` (which can only
extend it); the writable path memoizes in `SharedDomains`, reset at
flush and on `ViewID` change. A compile-time assert guards each memo's
bitmask against domain-count growth.

Positive entries are stamped with their step-derived txNum bound. A
negative entry is stamped with the last txNum its read view included,
`max(frontier-1, 0)`: since `Unwind(N)` treats N as the first
rolled-back txNum, the negative survives `Unwind(N)` and is invalidated
by `Unwind(N-1)`.

Admission safety needs one more invariant: a view's frontier never
decreases in a process that fills the cache (the DB component is frozen
at tx begin; a files reopen only extends it). This is enforced, not just
documented: wiring a fill-enabled `StateCache` forbids visibility
lowering on that aggregator, and `recalcVisibleFiles` panics on the one
transition that breaks admission — a cached state domain's visible end
decreasing — whichever entry point causes it. Raising visibility
(unaligning a lagging entity) stays allowed, and apply-only caches
(`STATE_CACHE_FILLS=false`) skip the forbid: with no fills there is
nothing for a lowered frontier to poison.

`Apply` physically deletes account, storage and code entries; on the
authoritative side absence is represented by absence, with no deletion
markers (a marker can be evicted, re-opening the fill window).
Reader-side negative entries do exist — they are ordinary
admission-gated fills, stamped as below. An account update invalidates
the derived address→codeHash mapping; an account deletion also removes
the address's code binding; an account-only deletion does not advance
the code-domain frontier (that would suppress valid code fills for
unrelated contracts). On authoritative code apply, bytes are cloned
before hashing so cached code and its codeHash cannot diverge.
`SeedAddrCodeHash` accepts only view-sourced account records (a
cache-sourced record can lag the latest apply and would carry pre-apply
state past the gate); a read error does not seed the zero-hash sentinel.

- **Reads are not snapshot-isolated**: a `ReadView` hit can be newer
than the view's tx. That is the same direction the Overlay already
serves (embedded RPC at `latest` reads exec's newest state on purpose),
and a single-version LRU cannot give stable per-view reads without
becoming a second kvcache — `node/shards` already provides that model
for the remote daemon. The cache's contract, in the forward direction,
is monotonicity: content never regresses behind the applied frontier;
unwinds invalidate by epoch and floor.
- **The reverse direction around unwind** — a view opened before an
unwind refilling a value from the discarded fork — is tracked separately
in #22463.

- Cache hits are unchanged: no frontier query, no admission lock.
- The memoized frontier read is `~2.4 ns/op`, 0 allocs (Apple M2 Max);
the memo adds `+64 B/op` to a read-only temporal tx
(`BenchmarkBeginTemporalRo/WithBlockSnaps`; absolute numbers drift with
unrelated changes on `main`).
- Getter paths (exec workers, RPC) build their frontier-carrying
`ReadView` once per getter; frontier-less views elsewhere
(`GetCode`/`GetCodeSize` fast paths) are by-value and allocation-free,
and a frontier is bound — one allocation — only on cold fill/seed paths.
A cold negative fill through the plain `GetLatest` wrappers is `~0.3
µs/op`, 48 B/op, 2 allocs: one allocation is the fill itself, one is
binding the frontier on the miss path — both amortize against the
backing read they follow (the no-cache baseline of the same read is
`~0.1 µs`).

Production-path reproductions through embedded RPC
(`execmodule.Cache.View` over a published `SharedDomains` overlay and a
real temporal DB): account, storage and code resurrection from an old
RPC read view, and code-of-a-deleted-account refill (pinning both the
SD-level paired deletion and the cache-level accounts-frontier check on
code fills). Plus: admission at the exact frontier boundary and a
20,000-round concurrent apply/delete vs fill interleaving; physical
deletion and derived code-mapping invalidation; account-only deletion
not suppressing unrelated code fills; the address→codeHash mapping
seeding only from view-sourced records; exact file/DB frontier
calculation and memo re-derivation after files-view reopen and after
flush; read-ahead freshness, no-exact-frontier backends never filling;
negatives surviving `Unwind(N)` but not `Unwind(N-1)`; the admission
frontier surviving `Clear`; the `STATE_CACHE_FILLS=false` apply-only
switch; `USE_STATE_CACHE=false` constructing no cache; node close
releasing every cache-budget reservation (a full EngineApiTester
lifecycle); and the visibility-lowering assert. All cross-package tests
exercise the public `ReadView`/`Applier` API. The three resurrection
tests use no API introduced by this PR: ported unchanged to `main`, all
three fail with the resurrected values served — they pin the bug, not
the refactor.

Verified with `go test` (and `-race`) on `execution/cache`,
`execution/exec`, `db/state/execctx`, `db/kv/temporal`, full-tree build,
benchmarks with `-benchmem`, and repeated clean `make lint` runs.

Suggested order:

1. `execution/cache/cache.go` (package doc) and
`execution/cache/view.go` — the contract and the API: `Frontier`,
`ReadView`, `Applier`. Small files; everything else implements them.
2. `execution/cache/state_cache.go` — admission internals: `appliedEnd`,
`fillIfFresh`, `apply`.
3. `db/state/execctx/domain_shared.go` — wiring: getters hold a
`ReadView`, commit/unwind hold the `Applier`; `getLatestMetered` shows
the full read chain (mem → cache → backing tx → fill).
4. `execution/exec/blocks_read_ahead.go` — the read-ahead fill path.
5. `db/state/execctx/statecache_rpc_integration_test.go` — the
resurrection reproduced end-to-end through the embedded-RPC path, and
the fix pinned.

---------

Co-authored-by: Alexey Sharov <askalexsharov@gmail.com>